Check allocatedObjects after script execution

Hi, here I faced so called "black magic code" which shows the allocated objects before and after script execution:

int *::+(int *ptr, int ofs)
{
    int *rtn = ptr
    rtn += ofs
    return(rtn)
}

int *::@(int *ptr, int ofs)
{
    int adr = *(ptr + ofs)
    int *rtn = addr_(adr)
    return(rtn)
}

int *mbn(int *ptr)
{
    return(ptr @ 0x74)
}

int *nnn(int *ptr)
{
    return(ptr @ 8)
}

int *ccp()
{
    DB db = create("")
    int *ptr = addr_(db)
    int *rtn = ptr @ 48
    destroy(db)
    return(rtn)
}

int allocatedObjects()
{
    int cnt = 0
    int *mb = mbn(ccp())
    while(!null mb) { mb = nnn(mb) ; cnt++ }
    return(cnt)
}

We have to thanks @Mathias Mamsch which is the author. 

So I run the function before and after my script ends and have the following result:

print allocatedObjects()

Before script alocationObjects: 0

After script allocationObjects: 69

Does it means that despite of the delete functions which I have used I have 69 objects which leaks the memory?

Re: Check allocatedObjects after script execution
Mike.Scharnow - Mon Jan 04 08:49:09 EST 2016

That might be the case. insert more lines of "print allocatedObjects()" to narrow down whether the code still works.

Note that this code is from 2012. So, it was developed for and most probably only works with 32 bit versions of DOORS!!

Re: Check allocatedObjects after script execution
Mathias Mamsch - Mon Jan 04 14:16:42 EST 2016

No this means, that after the script has run, there are 69 Objects that are still allocated, that will be deallocated automatically when the script ends, therefore not leaking memory.

However imagine inside the script you got a mainloop that iterates over a module with 20 objects and the script leaks 3 items per object (60 for the whole module) and maybe 9 global variables, which you can completely ignore. You will have no problem for small modules. But when you run your script over a modul with say 1000 objects, you will leak 3000 objects in the process, which will considerably slow down allocations in the process. That means when the script starts it might be able to process 100 objects a second, but when it ends it might only be able to process 10 objects a second, i.e. the script gets slower.

You need to understand, that a leaked object, will slow down ANY following allocation and deallocation. That means a perfectly valid function

string quoteString (string s) {
    Buffer buf = create(); 
    buf = "\""
    buf += s
    buf += "\""
    string result = stringOf buf
    delete buf
    return result
}

will become increasingly slower, because it contains an allocation (buf = create()) and a deallocation (delete buf). Therefore I consider it good practice to factor out the buffer to a global variable:

Buffer quoteString_buf = create(); 
string quoteString (string s) {
    quoteString_buf = "\""
    quoteString_buf += s
    quoteString_buf += "\""
    string result = stringOf quoteString_buf
    return result
}

although not everyone shares this opinion (globals are bad and stuff). Regards, Mathias

Re: Check allocatedObjects after script execution
Dinio - Mon Jan 04 15:36:35 EST 2016

string quoteString (string s) {
    Buffer buf = create(); 
    buf = "\""
    buf += s
    buf += "\""
    string result = stringOf buf
    delete buf
    return result
}

Buffer quoteString_buf = create(); 
string quoteString (string s) {
    quoteString_buf = "\""
    quoteString_buf += s
    quoteString_buf += "\""
    string result = stringOf quoteString_buf
    return result
}

So @Mathias_Mamsch 85b3aa38-6d03-41f0-8c83-a441b8c46b92, 

1. If I understand correctly, that there will no memory leak while my script ends?So if at the end even string memory will be free?​ When I start the script for the second time it shows the same numbers 0 and 69, not starting from 69 and 138 why....
2. If I do a lot of iterations over a lot of objects the leaking objects multiply and at the end of the script the Doors have to do a big time of deallocations and allocations during script run. SO it will be slow.
3. What allocatedObjects()  returns? Number of objects which have been created(memory has been allocated for them)? Or not deleted memory?

Re: Check allocatedObjects after script execution
Mathias Mamsch - Mon Jan 04 17:08:07 EST 2016

1. Strings are a different manner. Strings do not count as "Allocated Objects". String table content will persist. However I showed in another thread, that strings that are used only inside DXL (like concatenation, etc.) and not being passed to perms will be deleted from the string table when the DXL ends. Permanent string table entries will persist.

For the mere reason of not leaking allocated objects when the DXL script ends, DXL keeps a single linked list of all items, that are allocated on the heap. This list can be found in the Current DXL Context (ccp = current context pointer) - which also means each DXL script has its own list of allocated objects. This linked list has items with a simple structure, a pointer to the allocated memory, a pointer to the destruction function to call to free the resources for the item and a pointer to the next node in the list (nnp = next node ptr). The code above simply locates the DXL context (through a temporary Dialog Box item, which has a pointer to it), from it gets the Memory Block Nodes list (mmb) and loops through the items of the linked list, to count the objects that DXL keeps track of. When the DXL script ends, then for cleanup the DXL engine will walk this list and call the destruction functions for each allocated item. By the way the above code is also available in more "speaking" form in the forum.

A few things are interesting about this: a) you can identify the type of the allocated items from the destructor function (each type has its own). b) You can remove stuff from the list if you want to leak an item permanently. c) You can modify the function pointer to point to any function in memory and use this to execute your own machine code, as I showed in another post :-) d) The reason why DOORS gets slower and slower is simply due to the fact, that DOORS appends to the end of the list (and therefore walks the list every time) when making an allocation instead of inserting newly created items at the start of the list (which would probably eliminate the slowdown in 95% of the cases! duh!) e) There are a couple of DXL types like Stream, etc. that do not have a DXL destructor but will be freed when the DXL ends. These are good candidates to call in eval_ (which will make a child context and therefore cleanup the items when it ends).

So you see, that the allocations for each DXL context are by definition fresh at start, its not a global thing like the string table, its bound to the current DXL context.

2. Yes. While allocations are very fast at the start of a DXL script (30 microseconds or such), they can slow down to over a millisecond when the list of allocated objects grows longer. So this is not a linear slowdown but a quadratic one - why? I never found out.

3. Simply the number of the items in the linked list (remember each item points to the heap memory of some DXL variable that has not been freed yet)

Regards, Mathias

Re: Check allocatedObjects after script execution
Dinio - Tue Jan 05 06:26:43 EST 2016

Thanks a lot. But one more question.

1. If for every DXL script we have such kind of list, which contains pointer to the destructor function why it is necessary to call explicitly delete(object)?

Re: Check allocatedObjects after script execution
Mathias Mamsch - Tue Jan 05 06:50:54 EST 2016

1. It isnt (when your script can live with deleting all allocations at the end and does not suffer from slowdown)

However

a) a lot of times you will be doing library code. On library code, you will never know whether or not the user will need to watch out for allocations, so there I would consider it mandatory to cleanup everything.

b) Under certain circumstances your DXL Context will stay alive after it stops running, i.e. when you install a trigger or when you open a GUI. Closing the GUI incorrectly might also cause memory not to be freed.

c) Its good style to do so. You never know who will reuse your code someday.

Regards, Mathias